Machine virtualization is a technology for presenting the hardware of a computer as multiple virtual machines (VMs) each able to concurrently execute on the same computer. Each such virtual machine (VM) is able to host a guest operating system and the virtualized presentation of the computer's hardware to a guest operating system is mostly transparent to the guest operating system. Machine virtualization has become efficient, reliable, and convenient to manage. Consequently, machine virtualization is being used more often and for expanding types of applications and workloads.
The growing use of machine virtualization has led to increased demand for understanding the execution of a VM, analyzing performance, maintaining a record of a VM's computing activity, and debugging execution of a VM and/or its guest software. VM execution tracing is one technique that has suggested for these purposes. Execution tracing involves tracing the execution activities of a VM and storing the execution trace for later analysis. An execution trace can be used for detailed post-execution analysis and debugging of VM execution. Ideally, an execution trace is deterministic and “replayable”, i.e. it has sufficient information to enable playback features commonly found in software debugging tools such as stepwise execution of statements, reverse playback, detailed stack and symbol analysis before and after each statement, etc.
Although VM execution tracing has been recognized as desirable, previous attempts have had shortcomings in implementation and performance. One approach has been to force a VM to execute in a single thread of execution, which facilitates determinism but is difficult to implement and limits performance. For many types of guest software for which parallelism is critical, this limitation prohibits production use. Another approach has been to capture all of a VMs memory, but this carries startup and storage costs. Yet another approach has been to use complex run-time monitoring of memory states to maintain trace determinism, but this is difficult to implement correctly, has high overhead, and can significantly impact the performance of some workloads. Instrument guest software with trace-enabling instructions is another possibility but this has the practical drawback of requiring recompiling and deploying new executable code. Others have tried to interpose trace logic between a VM and a hypervisor, but with mixed success and many platform specificities.
What is needed are techniques for replayable VM execution tracing that are convenient to implement at the virtualization layer, have a low impact on VM performance, maintain causality for concurrent multiprocessing VMs, and can be used with or without VM visibility or actions.